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SUMMARY 


The  major  thrust  of  this  project  has  progressed  along  three  different  fronts.  1)  The  laser 
interferometer  has  been  constructed  and  a  serious  attempt  has  been  made  to  replicate  Williams’ 
studies  of  aliasing  with  a  similar  apparatus.  Surprisingly,  these  results  have  generally  failed  to 
replicate;  current  investigations  center  around  various  avenues  which  may  offer  an  explanation 
for  this  discrepancy.  2)  A  complementary  approach  has  been  started,  wherein  a  small  patch  of 
the  retinal  mosaic  is  mapped  using  small  points  of  light  Initial  results  are  quite  promising,  but  it 
is  clear  that  better  control  over  eye  position  will  be  necessary  if  this  is  to  be  a  practical  tech¬ 
nique.  The  use  of  video  imaging  to  monitor  fixation  is  under  study,  with  encouraging  results.  3) 
The  effect  of  eye-movements  on  an  aliased  image  was  studied  by  simulation.  Small  eye- 
movements  (ocular  tremor)  will  make  aliased  images  move  dynamically,  but  it  does  not  appear 
that  they  will  render  aliasing  patterns  invisible.  Simulations  of  the  embryonic  growth  of  the 
retinal  lattice  are  being  studied,  to  see  if  they  will  produce  lattices  with  the  same  sort  of  near¬ 
perfect  symmetry  that  is  observed  in  anatomical  lattices. 

In  a  brief  investigation  of  a  fortuitous  observation,  the  possibility  that  subjective  contours 
(i.e.  without  luminance  change)  could  trigger  stereopsis  was  studied.  It  is  possible  to  achieve 
stereopsis  without  objective  luminance-edges,  but  the  effect  is  extremely  weak. 


I.  Outline  of  Research  Activity 

A.  Proposed  Research 

Although  the  proposal  does  not  schedule  work  on  a  yearly  basis,  the  following  is  a 
reasonable  estimate  of  what  we  expected  to  do  in  the  first  year. 

1.  Construct  the  proposed  laser  interferometer.  This  is  basic  to  almost  all  of  the  remain¬ 
ing  work. 

2.  Reconcile  our  earlier  results  with  those  of  Dave  Williams  and  his  co-workers.  A 
major  part  of  this  proposal  was  based  on  the  assumption  that  by  building  an  apparatus 
identical  to  his,  we  could  both  replicate  his  results  and  --  by  simulating  our  earlier 
experiments  on  the  new  apparatus  —  determine  the  reason  for  the  long-standing  dis¬ 
crepancies  between  our  observations. 

3.  Begin  measurements  of  contrast  sensitivity  with  interference-fringe  gratings  over  a 
range  of  retinal  eccentricities.  This  relatively  long  study  would  be  useful  a)  as  ex¬ 
ploratory  parametric  data  in  a  new  area,  b)  it  would  contribute  to  resolving  the  above- 
mentioned  discrepancy,  and  c)  it  would  be  a  first  step  in  demonstrating  neural  alias¬ 
ing,  which  we  expect  would  manifest  itself  as  a  particular  pattern  of  sensitivity  (de¬ 
scribed  in  the  proposal). 


B.  Research  Accomplishments 

1.  The  Interferometer 

The  first  objective  has  been  the  construction  of  the  laser  interferometer.  This  was  begun 
in  January  ’89,  when  the  majority  of  the  components  had  been  accumulated,  and  proceeded 
rapidly.  It  was  clear  within  several  weeks  that  we  could  project  fringes  on  the  retina,  and  that 
problems  of  vibration  isolation  and  the  like  had  been  adequately  dealt  with.  A  plan  of  the  inter¬ 
ferometer  is  shown  in  Figure  1.  The  design  is  a  new  one,  provided  to  us  by  Dave  Williams,  who 
was  building  a  similar  interferometer  at  about  the  same  time.  This  design  —  though  comparable 
to  Dave’s  earlier  device  -  offers  a  distinct  advantage  in  that  the  two  interfering  beams  traverse 
exactly  the  same  optical  components,  only  in  opposite  directions.  This  cancels  out  much  of  the 
effect  of  vibration. 

The  interferometer  is  built  on  a  2’x4’  optical  table  which  rests  on  a  mattress.  This,  in 
turn,  rests  on  a  table  built  of  heavy  structural  aluminum  which  stands  on  a  concrete  floor.  The 
table  has  wheels  for  convenience  (the  device  weighs  many  hundreds  of  pounds),  but  these  are  by 
far  the  weakest  point  in  the  vibration  isolation.  In  use,  therefore,  the  interferometer  table  is 
placed  on  cement  blocks.  The  subject  is  stabilized  by  a  bite  bar,  mounted  on  a  2"x2"  aluminum 
column.  The  column  is  held  in  a  large  machinist’s  vice  with  two  orthogonal  lead  screws  which 
allow  us  to  precisely  position  the  subject’s  head.  The  weakest  link  in  this  chain  appears  to  be  the 
subject’s  dentition;  we  think  a  positive  forehead  rest  will  substantially  improve  this,  and  this  is 
under  design. 
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Figure  1  . 


The  interferometer  is  actually  assembled  on  2"  wide  bench  rails.  While  not  necessary 
(Dave  Williams  doesn’t  use  them)  these  provide  easy  attachment  of  devices,  with  easy  motion 
along  the  optical  axes.  The  straightness  of  these  rails  has  proven  entirely  adequate.  We  have 
copied  the  rails  in  acrylic  plastic  (along  with  their  component  holders)  for  use  in  non-critical 
parts  of  the  device.  The  most  difficult  part  of  the  construction  was  the  motorized  mirror-mounts, 
which  tilt  the  mirrors,  varying  the  orientation  and  spatial  frequency  of  the  grating.  While  such 
devices  are  commercially  available,  these  are  typically  1)  extraordinarily  expensive,  2)  several 
orders  of  magnitude  more  accurate  than  we  need,  and  3)  far  too  slow  to  run  real-time  experimen¬ 
tal  trials.  We  removed  the  stepper  motors  from  two  floppy  disk-drives,  attaching  them  to  the 
remains  of  the  drives  with  long  cables.  The  drives  remain  interfaced  to  the  computer  in  the 
normal  fashion.  This  allows  us  to  control  the  stepper  motors  using  both  the  hardware  and  soft¬ 
ware  from  the  original  disk  interface;  a  substantial  saving  in  interfacing  effort.  The  steppers  are 
geared  down  to  run  angular  mirror  positioners.  An  S-100  computer  serves  as  a  dedicated  proc¬ 
ess  controller,  running  both  the  steppers  and  the  acousto-optic  modulators. 

Unfortunately,  some  problems  have  arisen  around  the  interferometer.  First  of  all,  the 
start  of  serious  research  was  delayed  about  three  months  by  an  extraordinary  delay  in  the  deliv¬ 
ery  of  the  acousto-optic  modulators.  Without  these,  it  was  not  possible  to  vary  the  contrast  of 
the  fringes,  which  is  an  essential  part  of  our  planned  experiments.  A  second  problem  became 
apparent  simultaneously  to  us  and  to  Dave  Williams;  the  interferometer  depends  critically  on 
the  ability  of  polarizing  beam-splitter  cubes  to  separate  the  beam  into  opposing  polarizations 
with  >  99.9%  purity.  The  cubes  we  are  using  fall  very  short  of  their  specifications  in  this  regard. 
The  result  of  this  crosstalk  between  the  polarized  beams  is  that  we  cannot  reduce  the  contrast  of 
die  fringes  below  about  4%.  So  far  this  has  actually  not  affected  our  experiments,  and  we  have 
finally  located  a  source  of  adequate  beam-splitters,  so  this  problem  is  under  control. 

A  more  serious  problem  is  apparently  not  of  a  technical  nature,  however.  We  assumed 
that  when  we  had  duplicated  Williams’  apparatus,  we  would  be  able  to  replicate  his  experimen¬ 
tal  findings,  but  this  is,  in  fact,  not  the  case.  The  original  discrepancy  between  our  results  was 
this:  working  with  images  on  a  CRT  (rather  than  fringes)  we  found  that  orientation  discrimina¬ 
tion  at  7°  parafoveal  ceased  at  about  10  c/deg,  while  Williams  was  able  to  discriminate  orienta¬ 
tion  up  to  about  25  c/deg.  The  issue  is  an  important  one,  since  the  discrimination  limit  is  the 
operational  measure  of  the  onset  of  aliasing.  Thus  we  appear  to  be  observing  with  a  lattice 
whose  spacing  is  less  than  half  the  spacing  of  Williams’  lattice.  How  this  is  possible  is  by  no 
means  clear,  but  in  our  proposal,  we  observed  that  some  important  issues  hang  on  this  discrep¬ 
ancy,  and  might  be  elucidated  by  a  thorough  investigation.  Even  after  considerable  thought, 
both  in  Durham  and  in  Rochester,  no  one  has  a  very  plausible  suggestion  why  the  two  appara¬ 
tuses  yield  such  different  results. 

Despite  the  use  of  interference  fringes,  our  measurements  of  discrimination  limit  in  the 
parafovea  remain  quite  similar  to  our  earlier  measurements  (i.e.  they  are  at  least  a  factor  of  2 
different  from  Williams’).  Moreover,  we  have  been  unable  to  replicate  Williams’  published 
results  on  foveal  aliasing  ("zebra  stripes").  Above  about  60  c/deg  our  fringes  become  undetect¬ 
able,  and  nothing  else  is  seen  at  any  higher  frequency.  We  are  not  alone  in  this;  at  least  one 
other  group  has  seriously  tried  to  replicate  Williams  and  failed  (personal  communication;  re¬ 
searcher  wishes  to  remain  anonymous).  We  have  discussed  the  problem  with  Dave  Williams  at 
some  length,  but  no  resolution  has  come  from  these  discussions.  While  he  acknowledges  that 
the  fringes  are  subtle  and  hard-to-see,  about  80%  of  his  subjects  are  able  to  see  them.  Part  of  our 
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problem  may  be  a  practical  one;  to  view  fringes  above  about  80  c/deg,  it  is  necessary  to  dilate 
the  subject’s  pupil.  We  do  not  have  available  the  kind  of  medical  supervision  which  would 
permit  us  to  do  this  with  subjects  other  than  the  experimenters.  Perhaps,  by  misfortune,  Dr. 
Swift  and  I  are  among  the  20%  of  subjects  who  do  not  see  these  phenomena.  We  are  working  to 
get  authorization  to  use  other  subjects,  but  permission  to  administer  drugs  is  not  given  lightly  (or 
quickly). 

Part  of  our  difficulty  in  seeing  aliased  patterns  surely  results  from  the  extreme  noisiness 
of  the  interferometer’s  display.  An  inherent  problem  with  imaging  systems  using  coherent  light 
is  that  any  speck  of  dirt  in  the  optical  path  makes  a  distinct  image.  This  is  quite  different  from 
conventional  imaging  systems  where  the  vast  majority  of  such  intrusions  -  being  far  from  image 
planes  —  are  so  out  of  focus  as  to  be  invisible.  In  fact,  we  have  managed  to  keep  the  optics  of 
the  interferometer  quite  clean;  most  of  our  noise  is  entoptic,  either  free-floating  in  the  vitreous 
humor,  or  located  on  the  various  optical  surfaces  in  the  eye.  It  is  possible  to  suppress  the  noise 
from  one  of  these  surfaces  (the  outer  surface  of  the  cornea  is  the  worst  offender)  by  focussing 
the  laser  beam  exactly  on  that  surface,  but  this  still  leaves  a  very  noisy  image,  indeed.  The 
masking  effect  of  visual  noise  is  well-known,  and  it  may  be  this  which  is  obscuring  the  relatively 
faint  aliasing  patterns.  Entoptic  noise  is  believed  to  become  worse  with  age  (this  is  certainly  true 
for  the  author),  so  the  use  of  younger  subjects  may  be  helpful.  Thibos  has  done  extensive  stud¬ 
ies  of  aliasing  using  the  Lotrnar  white-light  interferometer.  He  has  been  quite  vocal  about  the 
advantages  of  white-light  interferometry,  precisely  because  the  noise  problem  is  almost  entirely 
avoided.  Our  experience  has  made  us  a  good  deal  more  impressed  with  Thibos’  viewpoint;  we 
are  considering  the  purchase  of  a  Lotmar  (about  $3000)  with  the  express  purpose  of  performing 
a  side-by-side  comparison.  Although  Williams’  interferometer  may  have  some  notable  limita¬ 
tions,  it  appears  to  us  that  entoptic  noise,  which  is  primarily  high-spatial-frequency,  is  less  of  a 
problem  in  the  parafovea,  where  acuity  is  reduced.  We  are  therefore  proceeding  to  take  para¬ 
metric  data  on  contrast  sensitivity  in  the  parafovea,  albeit  with  some  trepidation,  since  the  unex¬ 
plained  2-fold  difference  between  our  results  and  Williams’  is  still  present 

2.  Mapping  the  Retinal  Mosaic 

This  overall  project  deals  with  a  variety  of  indirect  methods  for  studying  the  retinal 
mosaic,  either  receptoral  or  neural.  All  of  these  techniques,  however,  approach  the  problem 
from  the  spatial-frequency  aspect  (e.g.  the  measurement  of  Nyquist  frequencies).  Given  the 
difficulties  described  above,  we  wondered  whether  it  would  be  useful  to  work  in  the  spatial 
domain  instead,  by  directly  mapping  the  functional  mosaic  using  the  detectability  of  small  points 
of  light.  The  lattice  is  too  dense  relative  to  the  large  point-spread  function  to  do  the  mapping  in 
the  fovea  (Jennings  and  Charman,  1981),  but  in  the  parafovea  the  lattice  density  falls  very 
sharply,  while  optical  quality  is  only  slightly  reduced.  We  recently  (Smith  and  Cass,  1989) 
published  data  on  spatial  summation  in  the  parafovea,  which  suggest  that  summation  areas  are 
about  3’  in  diameter.  Our  measurements  of  the  Nyquist  limit  (Smith  and  Cass,  1987)  offer  a 
direct  measurement  of  parafoveal  lattice  spacing,  which  is  also  about  3’.  While  these  figures  are 
certainly  imprecise,  they  are  accurate  enough  to  suggest  two  things.  1)  Both  the  summation  area 
and  the  lattice  spacing  are  considerably  larger  than  the  optical  point-spread  function  (about  1.2’), 
suggesting  it  might  be  possible  to  probe  these  structures  with  tiny  points  of  light  2)  Since  their 
separation  and  diameters  are  about  the  same,  summation  areas  will  be  relatively  non-overlap¬ 
ping;  thus  their  individual  properties  might  be  detectable  by  such  probes.  (On  the  other  hand, 
the  parafoveal  cone  mosaic  —  with  cone  diameters  of  1.5’  and  separations  of  2.3’  (Hirsch  and 
Miller,  1987)  —  is  probably  too  fine  for  probing  in  this  way.) 


If  we  tentatively  accept  that  a  point  stimulus  can  be  placed  on  a  summation  area  with 
reasonable  accuracy,  and  if  we  further  accept  that  the  summation  areas  described  above  have 
properties  similar  to  ganglion-cell  receptive  field  centers,  then  some  very  interesting  experiments 
become  possible.  The  simplest  idea  is  that  we  could  map  the  sensitivity  of  a  small  area  of  the 
retina  with  near-threshold  spots.  Such  an  experiment  might  show  a  pattern  of  small,  compact 
areas  of  high  sensitivity,  with  lower  sensitivity  between.  Some  initial  experiments  with  this  veiy 
simple  technique  have  not  yielded  statistically  significant  results,  but  this  does  not  surprise  us 
The  calculations  of  size  and  separation  given  above  suggest  that  the  sensitive  areas,  if  not  over¬ 
lapping,  are  probably  too  closely  adjacent  for  easy  separation.  A  more  sensitive  test  was  devised 
from  the  observation  that  the  majority  of  ganglion  cells  (the  midget  ganglion  cells,  about  85%) 
are  opponent-color-tuned.  We  changed  our  task  from  simple  detection  to  the  discrimination  of 
colored  spots  to  effectively  "thin  out"  the  mosaic,  since  a  colored  stimulus  will  stimulate  only 
certain  retinal  areas.  Moreover,  we  asked  the  subject  to  discriminate  complementary-colored 
spots  (red  vs.  green).  Since  the  same  summation  area  would  presumably  not  be  responsive  to 
both  colors,  we  hoped  to  find  a  negative  correlation  between  the  sensitivity  patterns  of  the  two 
colors.  We  have  tested  three  subjects.  One  appears  to  lack  the  fixation  ability  to  perform  this 
experiment  (see  paragraph  below).  A  second  subject  typically  gives  the  desired  data,  at  high 
statistical  significance  (p  c.001)  for  about  half  of  the  sessions.  A  third  subject  does  not  produce 
evidence  of  "gaps"  between  receptive  fields.  However,  this  was  the  same  subject  (PC)  who 
showed  the  smallest  receptive-field  separations  in  our  previous  measurements  of  the  parafoveal 
Nyquist  limit  (Smith  &  Cass,  1987).  This  partial  success  encourages  us  to  think  that  there  is  a 
real  effect  here,  which  more  refined  techniques  may  reveal.  Some  representative  data  are  shown 
in  Figure  2. 

A  major  methodological  problem  with  these  studies  is  the  questionable  ability  of  our 
subjects  to  maintain  fixation  with  the  necessary  accuracy  (perhaps  2’).  The  subject  is  provided 
with  a  fixation  cross,  and  is  allowed  to  trigger  each  trial,  selecting  moments  when  his  fixation 
seems  best.  There  are  no  published  data  indicating  how  good  fixation  might  be  under  these 
conditions,  though  Riggs,  Armington  and  Ratliffs  (1954)  data  suggest  that  the  desired  accuracy 
is  possible.  Nonetheless  we  are  certain  that  fixation  error  is  adding  considerably  to  the  noise  in 
our  data,  and  may  be  accounting  for  the  lack  of  effect  for  certain  subjects  and  for  certain  ses¬ 
sions.  We  have  decided  to  attack  the  eye  movement  problem  directly,  by  monitoring  eye  posi¬ 
tion.  In  extended  discussions  with  Alan  Kielar  of  Iscan  Corporation,  we  find  that  they  are  able 
to  track  eye  position  to  about  8’  accuracy  in  real  time.  The  limitation  on  their  system  is  the  size 
of  a  single  pixel.  However  it  is  quite  possible  to  accurately  estimate  the  location  of  an  object  to 
a  fraction  of  a  pixel  (analogous  to  hyperacuity  ir  vision)  using  an  appropriate  algorithm.  Using 
digitized  photographic  images  of  the  eye  (Figures  3  -  5),  we  have  worked  out  algorithms  for 
tracking  both  the  corneal  reflection  and  the  pupil.  It  is  necessary  to  monitor  both,  since  the 
comeal  reflection  is  used  as  a  control  for  small  head  movements.  Both  algorithms  are  stable  to 
about  5"  under  small  perturbations  of  threshold  and  other  parameters.  While  this  is  not  a  proof 
that  the  algorithms  will  work  in  an  actual  experiment,  it  is  most  encouraging. 

In  practice,  the  experiment  we  propose  is  as  follows.  The  subject  will  perform  essential¬ 
ly  the  same  task  as  in  the  present  situation:  fixating  a  cross  while  determining  the  color  of  a 
briefly-flashed  spot  presented  7°  in  the  parafovea.  When  the  spot  is  presented,  a  frame-grabber 
will  digitize  an  image  of  the  pupil,  taken  in  either  continuous  or  strobed  infrared  illumination, 
which  will  then  be  processed  during  the  inter-trial  interval.  Note  that  we  do  not  propose  to  track 
the  eye  to  high  precision  in  real  time;  this  is  probably  not  possible  with  current  computer  tech¬ 
nology.  Rather  we  are  allowing  the  computer  several  seconds  to  process  each  frame.  With  an 
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accurate  localization  of  eye  position,  we  can  calculate  our  retinal  sensitivities  in  terms  of  where 
the  spot  actually  struck  the  retina,  rather  than  where  it  was  presented  on  the  display  screen.  We 
hope  that  this  mil  dramatically  improve  our  results. 


#  of  Correct  Blue  Responses 


#  of  Correct  Yellow  Responses 


Difference  ( Blue  -  Yellow) 


Figure  2a  Subject  fixated  carefully  and  discriminated  between  blue  and  yellow  dots  presented 
seven  degrees  in  the  superior  parafovea.  Each  square  in  the  matrices  represents  a 
single  point  of  light.  Center-to-center  spacing  between  points  was  about  four  arc- 
minutes,  Blue  and  yellow  spots  were  each  presented  30  times  at  each  location.  The 
squares  in  the  matrices  at  the  top  represent  the  number  of  correct  detections  for  each 
color.  The  matrix  at  the  bottom  represents  the  difference  of  the  two  upper  matrices. 
Note  that  blue  is  seen  more  readily  in  the  upper  right  quadrant;  while  yellow  is  seen 
better  in  the  iower  left.  Results  are  highly  statistically  significant. 
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Difference  ( Green  -  Red ) 


Figure  2b  Same  as  Figure  2a,  except  with  green  and  red  spots.  Retinal  location  of  spots  is 
similar  but  not  precisely  the  same  as  in  Figure  ^a.  Note  that  there  is  an  area  at  the 
bottom  middle  that  seems  to  more  sensitive  to  green  than  to  red.  While  this  may  not 
seem  convincing  visually,  the  correlation  between  red  detections  and  green  detec¬ 
tions  at  each  position  was  -.81,  and  was  highly  significant. 
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Figure  3.  This  grey-scale  rendition  of  an  eye  shows  the  corneal  reflection  (white  spot)  as  well  as 
the  pupil  (larger  dark  area). 


Figure  4.  Localization  of  the 
corneal  reflection.  The  circle 
shows  the  progress  of  the 
localization  algorithm. 


Figure  5.  Localization  of  the 
pupil.  The  first  estimate  of 
the  pupil  center  is  given  by 
the  center  of  a  brightness  cut. 


8 


3.  Modeling  the  Retinal  Mosaic 

We  undertook  modeling  of  the  retinal  mosaic  because  we  were  bothered  by  the  following 
question.  Retinal  aliasing  patterns  ("Zebra  stripes")  are  essentially  Moir6  patterns,  and  Moir6 
patterns  have  the  property  that  a  small  relative  movement  of  the  generating  patterns  may  pro¬ 
duce  a  large  movement  in  the  pattern  they  generate.  Specifically  if  either  of  the  generating 
patterns  moves  through  one  wavelength  (or  repetition  distance),  then  the  Moir6  pattern  will  also 
move  through  one  wavelength.  In  the  case  of  aliased  interference  fringes,  the  repetition  dis¬ 
tance  will  be  smaller  than  the  cone  separation,  while  the  wavelength  of  the  Moir6  fringe  may  be 
as  much  as  2  orders  of  magnitude  larger  than  that.  Thus  a  small  velocity  of  the  fringe  will  trans¬ 
late  into  an  enormous  velocity  of  the  perceived  pattern.  Since  the  eye  is  always  subject  to  small, 
high-frequency  tremors  whose  size  is  comparable  to  the  cone  spacing,  we  wondered  how  the 
Moir6  pattern  could  ever  remain  still  enough  to  be  seen.  We  therefore  decided  to  devise  a 
simple  model  in  which  we  could  investigate  this  question  at  least  semi-quantitatively. 

Although  there  are  considerable  technical  difficulties  in  assessing  fine  eye-tremor,  the 
best  available  data  (Eizenman,  Hallett  &  Frecker,  1985)  place  it  in  the  40  -  100  Hz  frequency 
range.  We  are  limited  in  the  temporal  resolution  of  our  simulation  by  the  60  HZ  frame  rate  of 
our  CRT.  Within  these  constraints,  we  decided  that  a  reasonable  worst-case  scenario  was  to 
assume  that  the  spatial  phase  of  the  interference  fringe  with  respect  to  the  lattice  is  random  when 
sampled  60  times  per  second.  We  modeled  this  by  generating  a  perfect  triangular  lattice  of 
receptors,  and  projecting  on  this  a  vertical  grating.  The  resulting  image  could  actually  be 
viewed,  and  it  displayed  aliasing.  We  calculated  a  number  of  these  images,  of  different  relative 
phases,  and  stored  them.  The  display  now  selected  images  at  random  to  present  every  1/60 
second.  Interestingly  such  a  procedure  did  not  produce  anything  like  perfect  cancellation  of  the 
successive  gratings.  Although  there  were  brief  periods  when  the  stimulus  was  invisible,  there 
were  also  periods  when  the  stimulus  remained  relatively  stationary,  and  the  aliased  (or  Moird) 
patterns  were  quite  visible. 

While  this  simulation  relies  heavily  on  machine-dependent  --  rather  than  biological  ~ 
parameters,  we  feel  that  the  known  parameters  of  ocular  tremor  are  not  so  different  from  our 
experimental  ones  as  to  invalidate  our  basic  conclusion.  That  conclusion  is  that  rapid  involun¬ 
tary  eye-movements  are  not  fast  enough  to  produce  cancellation  of  the  image  at  all  times,  and  so 
do  not  seriously  disrupt  the  potential  visibility  of  aliased  patterns.  As  we  expected  from  first 
principles,  however,  such  movements  do  cause  the  image  to  change  rapidly  and  dynamically. 

We  were  interested  in  what  effect  the  use  of  a  more  realistic  lattice  would  have  on  this 
simulation.  If  we  simply  allow  a  small  random  jitter  about  the  ideal  positions  of  the  receptors, 
this  produces  noise  in  the  aliased  pattern.  This  is  typical  of  the  parafoveal  lattice,  and  is  not  very 
surprising.  We  were  more  interested  in  the  nearly  regular  lattice  in  the  fovea.  Two  questions 
immediately  presented  themselves:  1)  how  do  we  accurately  characterize  the  almost-regular 
foveal  lattice,  and  2)  how  can  we  produce  similar  lattices  in  our  simulation?  We  attacked  the 
second  of  these  questions  first,  using  a  novel  approach.  We  reasoned  that  the  foveal  lattice 
results  from  a  large  number  of  embryonic  cone  cells  growing  to  fill  a  basically  fixed  space.  We 
therefore  modeled  this  growth  process  (though  we  found  it  technically  more  convenient  to  keep 
the  cones  the  same  size,  and  shrink  the  space  within  which  they  were  free  to  move  about).  A 
major  issue  in  this  simulation  turns  out  to  be  the  boundary  condition  at  the  edge  of  the  field  of 
receptors.  We  have  studied  rigid  and  elastic  boundaries,  and  boundaries  of  different  shapes 
(circular,  hexagonal).  A  hexagonal  boundary  is  much  more  likely  to  produce  hexagonal  paclong 
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than  a  circular  one,  for  example.  We  also  found  that  it  is  necessary  to  do  the  calculations  to  a 
high  precision.  If  calculations  are  rounded  to  the  nearest  screen-pixel,  then  the  underlying 
square  pixel  array  influences  the  simulation,  often  producing  a  square  lattice!  At  this  point,  our 
general  conclusion  is  that  the  effect  of  the  boundary  is  considerably  greater  than  we  could  wish, 
given  that  the  boundary  constraints  on  the  biological  system  are  by  no  means  clear.  We  are 
studying  more  general  and  flexible  constraints,  in  the  hope  that  these  may  prove  less  trouble¬ 
some. 


4.  What  Will  Fuse 

In  the  course  of  developing  the  pcStereoscope™  (which  is  quite  distinct  from  our 
AFOSR  project),  we  programmed  demonstrations  of  virtually  all  of  the  well-known  stereophe¬ 
nomena  -  probably  more  than  have  ever  before  been  assembled  using  the  same  observers  and 
the  same  apparatus.  In  the  course  of  this,  we  inevitably  noticed  some  unexpected  phenomena, 
some  of  which  seemed  worthy  of  purely  scientific  investigation.  One  of  these  is  the  issue  of 
whether  it  is  possible  to  fuse  subjective  (or  non-luminance-defined)  contours. 

The  classical  theory  of  stereopsis  was  that  an  object  was  localized  by  each  eye,  and  the 
relative  difference  in  position  is  interpreted  as  depth.  Julesz  (1971)  showed  that  this  story  was 
backwards.  Each  point  of  light  to  each  eye  is  correlated  with  points  of  light  to  the  other  eye. 
The  disparity  that  produces  the  highest  correlation  becomes  dominant,  and  all  points  at  that 
disparity  are  then  seen  as  a  form.  Julesz’  theory  has  become  well-established,  but  Julesz  never 
really  proved  that  the  old  theory  could  not  work  —  only  that  it  was  unnecessary.  The  current 
experiment  attempts  to  answer  the  question:  can  form  perception  be  the  basis  of  stereopsis?  To 
do  this  we  must  produce  stereograms  in  which  the  fusion  contours  are  monocularly  visible,  but 
have  no  luminance  change  associated  with  them. 

Four  separate  conditions  were  included.  All  share  the  following  features:  The  left  eye 
and  right  eye  each  see  a  square.  The  two  squares  are  offset  slightly.  On  the  left  side  of  the 
screen  the  disparity  is  uncrossed  (square  should  be  seen  behind  the  screen);  on  the  right  side  the 
disparity  is  crossed.  The  squares  seen  by  the  left  and  right  eyes  share  no  brightness  cues-their 
only  similarity  is  that  they  are  squares. 

Two  of  the  following  conditions  have  been  performed  before;  two  are  new.  A  brief 
description  of  each  follows: 

KANIZSA  SQUARES  -  Each  eye  sees  a  subjective  (or  illusory)  square  that  arc  produced 
by  different  inducing  figures.  On  the  left  side  of  the  screen,  one  eye  gets  pac-men  figures;  the 
other  gets  lines.  For  the  other  side  of  the  screen,  the  situation  is  reversed.  Therefore  the  induc¬ 
ing  figures  themselves  cannot  be  seen  in  depth.  The  only  basis  for  stereopsis  is  the  subjective  or 
illusory  squares. 
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Left  Eye 


I  I  I 


-  -  Right  Eye 
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In  the  above,  the  square  on  the  left  should  be  seen  in  crossed  disparity  (closer  to  the 
viewer,  the  one  on  the  right  in  uncrossed  disparity. 


STRUCTURE-FROM-MOTION  -  Each  eye  sees  a  square  that  is  formed  solely  by 
motion.  At  any  given  instant,  each  eye  is  simply  seeing  a  random  Field  of  dots.  However,  a 
square  portion  of  dots  changes  position  between  frames,  giving  rise  to  the  percept  of  a  moving 
square.  While  each  eye  is  seeing  the  same  square,  the  random  dots  presented  to  each  eye  are 
completely  different.  The  only  basis  for  stereopsis  here  is  the  forms  produced  by  the  motion 
system  (Lee,  1971). 


Time  1 


Left-eye  View 


Right-eye  View 


Time  2 


Left-eye  View 


Right-eye  View 


The  above  four  figures  represent  four  frames  of  the  CRT— left-eye  frame,  right-eye 
frame,  left-eye  frame  again,  and  nght-eye  frame  again.  The  two  inner  squares  on  each  frame 
have  been  highlighted  for  illustrative  puiposes  only.  Note  that  the  inner  squares  have  moved  up 
between  Time  1  and  Time  2.  Each  eye  therefore  sees  a  square  moving  upward.  Note  also  that 
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the  right-eye  squares  and  the  left-eye  squares  do  not  line  up  in  the  horizontal  domain.  This 
disparity  produces  the  depth  effect.  This  figure  is  a  simplification  of  the  actual  stimulus  in 
which  approximately  250,000  random  dots  were  used. 


TEXTURE  DIFFERENCE  -  The  squares  are  defined  by  their  texture  difference  with  the 
background  (Ramachandran  etal .,  1973).  Once  again,  however,  different  random  dots  are  used 
for  each  eye,  allowing  no  point-by-point  brightness  correlation. 


Left-eye  view 


Right-eye  view 


In  the  above  pair  of  figures,  the  square  on  the  left  side  of  the  screen  will  be  seen  in 
crossed  disparity  (closer  to  die  viewer). 
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VERTIC AL/HORIZON T AL  -  Here  the  squares  are  formed  by  a  pattern  difference  with 
the  background.  On  the  left  side  of  the  screen,  one  eye  gets  vertical  lines  and  the  other  gets 
horizontal  lines.  On  the  right  side  of  the  screen,  the  situation  is  reversed. 


Left-eye  view 


Right-eye  view 


In  the  above  pair  of  figures,  there  is  only  a  small  amount  of  disparity  (offset  of  the 
squares  to  the  different  eyes).  The  square  on  the  left  side  of  the  screen  will  be  seen  in  crossed 
disparity  (closer  to  the  viewer).  In  the  original  version,  the  mean  luminance  of  the  center  square 
and  the  surround  were  equal. 
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The  experimental  procedure  was  an  interwoven  forced-choice  staircase.  On  every  trial, 
the  square  on  one  side  was  presented  in  crossed  disparity,  and  the  square  on  the  other  side  in 
uncrossed  disparity,  under  conditions  of  randomization.  Each  condition  was  presented  20  times, 
and  the  subjects  has  to  determine  the  side  that  contained  the  "closer"  square  (i.e.  the  square  that 
was  presented  in  crossed  disparity).  Six  subjects  were  run  in  all,  and  the  experiment  was  self- 
paced.  No  feedback  was  given.  Disparities  were  approximately  6  arc-minutes  in  all  cases. 

The  results  for  the  individual  subjects  are  presented  in  Figure  6.  There  were  significant 
differences,  both  between  individuals  and  between  conditions.  The  combined  results  can  be 
seen  in  Figure  7.  The  results  for  the  vertical/horizontal  lines  produced  the  best  depth  effects, 
though  not  by  a  huge  margin.  This  may  be  due  to  the  fact  that  the  assimilation  phenomenon 
associated  with  thin  lines  produces  an  additional  possible  cue-a  perceived  brightness  difference 
between  the  squares  and  the  background. 

It  should  be  noted,  however,  that  all  conditions  produced  results  that  were  significantly 
above  chance.  The  major  conclusion,  therefore,  is  that  the  form  system  can  feed  the  stereopsis 
system,  albeit  weakly. 
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